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This report presents a brief summary of the full 
report of an evaluation of California schools receiving funds from 
three state and federal school funding programs in 197i(*75* The 
evaluation vas conducted by the California State Department of 
Education. Evaluated schools were funded through: (1) the Early 
Childhood Education (ECE) program, vhich provide state funds to 
schools vith students in kindergarten through third grade to increaise 
competency in reading, language, and mathematics; (2) the Elementary 
and Secondary Education Act (ESEA) Title I, vhich provides federal 
assistance for schools vith students from lov families; and/or (3) 
'the Educationally Disadvantaged Youth (EDT) program, vhich provides 
state funds for schools vith students vhose educational disadvantage 
results from lov family income, language barriers, and transiency. 
Sources of evaluation data included ratings of school plans, school 
self-reports, school visitations, and student achievement data. 
Besults of evaluation shoved that schools participating in the three 
funding programs evidenced positive change. (BBT) 
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Introduction to the Report 



The /evaluation Report of U% KSEA Title U 
and EDY, 1974-75, siiiiiniarizcd in this tloaniitjnt, 
is D comprclicnsivc report of the effects of local 
school progrunis, kinclergortcn througli twelfth 
grade, receiving slate and fedci:;il monies through 
the Consolidated application process* Tiie consoli- 
dated applieatioii, designed pnrsuatU to the provi- 
sions of Asscmhiy C'oncurrent Kcsohilion 127 
(l%9), provides a vehicle to briny together in a 
united planning effort the various state and federal 
supplementary funding sources, which were pa*- 
viously fragmented and often administered sepa- 
rately. 

This Summary of the evaluation report provides 
an overview of the tolal impact of the state funded 
early childhood education (ECI£) reform effort* the 
educationally disadvantaged youtl^. (EDY) pro- 
gram, and the federally funded Elementary and 
Secondary lidiication Act (ESEA) Title I program. 
These three largest of California's supplementary 
education efforts, which served 806,752 students 
in 1974-75, frequently overlap within a given 



school Bccuuse of this substantial degree of 
overlap, it lias become cicur that one consolidated 
evaluation report covering these three efforts is 
appropriate. Tlie evaluation report itself and a 
separate appendix provide-complete data coneeni- 
ing HCe, i;Sl£A Title 1, and HOY for 1974-75, 
*Bolh the report and the appendix are availahle 
upon recjucsl to the Department of Education, The 
appendix, wliicii was produced as a separate 
doeument, is an exhaustive compilation of the 
original data. 

This summary of the evaluation report has been 
arranged in much the same manner as the full 
report itself: (1) the progmm description for ECF., 
ESEA Title I, and EDY; (2) the procedures, 
instru mentation r and limitations of the study; and 
(3) the findings of tiie evaluation. However^ to 
assist the reader in gaining an immediate picture of 
the evaluationi the Department has prepared a 
summary of the findings and implications and has 
made that summary the first section of this 
document. 



Summary of Findings and Their Implications 



Tile findings presented in this section of tlie 
report were based on a review and analysis of the 
data gathered hy the State Department ot iiduca- 
tion regarding ttiose schools participating in the 
early childhood education {liCli) reform effort and 
those with LSliA 1itle I and edncationally dis- 
advantaged yonth (HDY) programs in 1974-75, 
The evaUiation findings are arrangeil accoriling to 
this order: institutional change: student achieve- 
ment; participants in I X l;, liStiA Title I, and IMJY; 
expenditure patterns, and tlic cvatuation proecss. 

Institutional Change 

livtdence of institutional ehange in schools 
within tue ECR refunn effort was gathered from 
the quality ratings of the school level plans, t|uality 
reviews condueted during tlic monitor and review 
(MAR) sehool visitations, and the product evalua- 
tion reports prepared by sehools in which they 
described and evaluated their programs at the end 
of the year. The changes in the schools (institu- 
tional change) were evident froni these 1^74-75 
Undings: 

• The p!aii rating information showed that all 
ttie schools were doing systematic planning, 
with }nore than 75 percent conducting (piality 
needs assessments. 

t Tlie plan rating information stioweil that 80 
pcreent of tlic schools were able |o write 
(tuality goals, anil 70 percent were able to 
develop i|uality objeetives. 

• The product evaluation report information 
showed a 54 pereent inerease in number of 
pitrtieipants lu |)arent parttcipalion compo* 
nent aetivities and a 55 percent inerease in the 
number of participants in parent education 
component activities over the 1973-74 data, 
with a 30 pereent increase in participating 
sclioots. 

t 1'he juonitor and review (MAR) data showed 
that of the 913 HCE schools visited, more 
than 87 pereent had individualized, diagnostic 
instruetional programs operating at or above 
the satisfaetory level in all phases of reading 
and mathematies. 

• The MAR d:Ua showed that more than 85 
percent of the sehools had statT development 



prognims which systematically were meeting 
the assessed needs of teachers, piiid aides, and 
administrators at or above the satisfactory 
level 

t The self-report data in the product evaluation 
report and in-depth study information indi- 
cated that local evaluation was occnnint;. 
The clear implication from these findings was 
lhat IXU schools were nvjking major clnmi^cs 
Ihroughout many areas of their programs, inilicat- 
ingin turn basic changes in the institutions. 

Three areas of institutional change, attliongh 
rated well, were snftlcicntly below a level of 
tjuality to warrant further examination: 

L More than 25 percent of the 913 IICK schools 
visited had some language development aro:ts 
related to hidividualizeil instruction which 
were below tlie satisfactory level. Additional 
data indicated a lack of clarity existed regard- 
ing the nieariing of language development, and 
the data also indicated that instructional 
materialsand measurement tools were lacking. 

2. Twenty-three percent of the 913 UCH schools 
visited were rated below the satisfactory level 
in parent participation in program evaluation. 
This iJidicated a need for more effort to 
involve parents in this specific activity* 

3. The inservicc programs in 27 i)crccnt of the 
liCii schools visited were rated below the 
satisfactory Icvc) in meeting the assessed 
iiecils of volunteers. This Unding indicated a 
need for increased attention to tins area, 
particulary since the number of volunteers in 
schools has increaiied so greatly. 

Student Achievement 

In student rcadhig and mathematics achieve- 
ment, the results of programs developed with all 
funding combinations cijualed or exceeded last 
year's achievement, which was an average g;iui of 
II months growth for 10 months in schtx^l. In 
addition, the pre^post standardized testing showed 
that: 

• ECE only schools attained reading achieve- 
ment above the national average on post-test 
scores in all grades served. 

• Tlie schools in the liCB reform effort tended 
to have higher scores in all achievement 
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areas than schools not included in the effort 
(ESEA Title 1 and/or EDY only). 

• Schools with ESEA Title 1 and EDY funded 
programs maintained month-to-month 
growth, when historically such schools would 
only be expected to gain seven months in ten 
months of instruction. 

• Schools entering ECE for the first time in 
1 974-75 had significantly greater achieve- 
ment, statistically, than the original schools in 
1973-74 in all areas except second grade 
mathematics. 

in addition the California state assessment test- 
ing showed that students in the ECE process 
achieved significantly higher, statistically, than did 
matched groups of students not in the program. 

Participants in ECE, ESEA Title I, and EDY 

A total of 806,752 students were served through 
the combined funding sources in 1974-75. Sixty- 
five percent of the students were enrolled in 
kindergarten through grade three (ECE, ESEA 
Title 1, and EDY); 23 percent, in grades four 
through six (ESEA Title I and EDY); and 12 
percent, in grades seven through twelve (ESEA 
Title I and EDY). More students received services 
in reading than in any other instructional compo* 
nent. A duplicated count showed that 770,000 
participating students were included in reading 
instruction, 580,000 in language development, and 
717,000 in mathematics. Large numbers of volun- 
teers were working in the programs: 67,000 adults 
contributed 200,000 hours per week, and 61,000 
students contributed 1 56,000 hours per week. 

Expenditure Patterns 

Examination of the final fiscal reports from a 
limited sample of districts showed differences in 
the patterns of expenditures within ECE, ESEA 
Title I, and EDY. In ECE, 55 percent of the funds 
went to pay classified salaries, and 21 percent of 
the funds were used for certificated salaries. In 
ESEA Title I programs^ 43 percent of the funds 
were used for classified salaries and 33 percent for 
certificated salaries. In EDY programs, 10 percent 
of the funds went to pay classified salaries, while 
71 percent of the funds were used for certificated 
salaries. The need for an analysis of al! the 
expenditure data was implied by this finding, and 
this analysis is currently being made. 

Evaluation Process 

As indicated h the section of the evaluation 
report entitled "Procedures, Instrumentation, and 



Limitations/' a variety of problems were identified 
in the evaluation instrumentation. The Department 
has already made the following changes for 
1975-76: 

1 . The plan rating instrument has been rede- 
signed to correspond more closely to the 
planning process. In addition calculations of 
inter-rater reliability are being made on the 
plan raters. 

2. In light of the concerns raised in 1974-75, the 
monitor and review (MAR) instrument has 
been redesigned to make it more functional 
and to allow separate measurements of imple- 
mentation according to the school's plan, 
progress toward restructuring or revitaliza- 
tion, and quality of the program. Measure- 
ments of inter-rater reliability and additional 
inservice training in the instrument's use are 
being pursued. 

3. The instrument used to assess program con^ 
pliance has been reduced to a more manage- 
able size, and it focuses on statutory require- 
ments. 

4. The progress implementation report has been 
eliminated, since it was a requirement which 
seemed to report data of little usefulness. 

5. The product evaluation report has been re- 
duced in size and will specify enumeration 
data, data from standardized test results, and 
data on the schools accomplishment of objec- 
tives. All standardized test results are to be 
reported in mean raw scores* 

6. In-depth studies, with less dependence on 
self-report data, will be done only for selected 
components and processes, emphasizing the 
processes of institutional change. 

7. Plans are being made to provide longitudinal 
data in the continuing evaluation process. 

There was difficulty in measuring the gains made 
by limited-English speaking students and non- 
English speaking students due to the lack of 
appropriate instrumentation. The Department of 
Education is currently engaged in developing such 
instrumentation, but it is not scheduled to be 
completed until 1977. The same problem of 
pre-post gain measurement will therefore exist in 
the 1975-76 report. 

Because the consolidated evaluation format pro- 
vided comprehensive program information instead 
of isolated information for each funding source, 
the continued use of this format is indicated for 
the 1975-76 consolidated evaluation report. This 
approach, with the previously specified modifica- 
tions, should provide even fuller data for 1975-76. 
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Program Description 



This section of the summary gives the reader a 
background against which to measure the effects of 
the three funding sources. It provides an overview 
of the enabling legislation for ECE, ESEA Title I» 
and EDY, followed by participant, funding, and 
eligibility requirements. The assumptions for these 
programs are presented, followed by an outline of 
the policy requirements. These requirements deal 
with institutional change and student achievement 

Enabling Legislation 

When the early childhood education (ECE) 
reform effort was enacted in Chapter 1402, Stat- 
utes of 1972, the California Legislature provided 
for a restructured primary education deigned to 
assure that all students in kindergarten through 
third grade would receive instruction that would 
meet their unique needs, talents, interests, and 
abilities. The Legislature called for the cooperation 
and extensive participation of parents and the 
community in the education of children in these 
early grades. The Legislature also asked that 
maximum use be made of existing state and federal 
funds in a coordinated effort to help primary 
school students increase their competencies in 
reading, language, and mathematics skills and thus 
heip ensure their achievement in later grades. 

The state funded program for educationally 
disadvantaged youth (EDY) was designed to pro- 
vide quality educational opportunities for students 
whose educational disadvantage had resulted from 
low family income, language barriers, and tran- 
siency. The EDY program was authorized in 
Chapter 1406 of the Statutes of 1972 (SB 90), 

Similarly, by enacting Title 1 of the Elementary 
and Secondary Education Act (ESEA) in 1965 
(P,L, 89-10 as amended), the United States Con- 
gress provided financial assistance for the augmen- 
tation of educational programs for students from 
low*income families. The California State Ekepart* 
ment of Education allocates and monitors basic 
grants of money to local educational agencies 
qualifying for the ESEA Title 1 funds. 

In developing a program, a local school may use 
one or ^ combination of federal or state monies for 
which it qualifies, as long as it can provide a proper 
accounting of the funds from each source. In 
addition to the three main sources cited, the 



following were also sources of funds fof the 
1974-75 programs: Miller-Unruh Basic Reading 
Act; Bilingual Education Act of 1972; ESEA Title 
Ih Phase I; and American Indian Early Childhood 
Education Act, 

PartictpantSf Funding, and Eligibility 

The 1974-75 state budget provided $40 million 
for the support of ECE, which involved about a 
fourth of Califomia*s student population in kinder- 
^ten through grade threet On approval by the 
State Board of Education of a school's proposed 
program, ECE money was granted for a school on 
the basis of $130 per student in kindergarten 
through third grade^ with an extra $65 each for up 
to 25 percent of the kindergarten through third 
grade students who were in the lowest quarter in 
achievement. 

In each year, half the ECE funds for any one 
participating district must go to those individual 
schools in the district which have the greatest 
educational need. Because ECE serves all children^ 
kindergarten through grade three in a school, ECE 
outcomes can be expected to be higher than those 
for ESEA Title I and EDY programs^ which serve 
only specially identified participants in each 
school. In ECE schools these special populations 
continue to receive Title I and/or EDY monies. 

The ECE legislation requires that each partici^ 
pating ECE school be evaluated and given a 
composite score based on a quantitative estimate 
of the degree and success of progt:am implementa- 
tion, pupil progress, and fiscal expenditures. Table 
1 shows the factors examined and their weights in 
each successive year of program implementation, 

California ESEA Title 1 funds in 1 974-75 totalled 
$155 million. These funds were allocated accord- 
ing to formulas based on census information and 
Aid to Families with Dependent Children (AFDC) 
data. 

Once ESEA Title I funds are allocated to a 
school, students to receive the services are selected 
on the basis of their educational need. All students 
scoring at or below the second quartile on stan- 
dardized achievement tests or those who have 
serious learning deficiencies because of linguistic, 
sociaU and cultural differences or economic isola*^ 
tion are eligible. In their use of ESEA Title 1 funds. 
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Table I 

Factors Rated in ECE Schools, with Weigltts 
Assigned Each Factor 



Factor rated 


Weight given to facior rated, 
expressed as a percent, 
by year of participation 


First 
year 


Second 
year 


Third 
year 


E)egree and success of program 








implementation 


70 


50 


50 


Quantitative estimate of 








pupil progress 


10 


40 


50 


Fiscal expenditure 


20 


10 


0 



districts must provide extra services to ESEA Title 
1 participants over and above what they provide to 
nonparticipating students. Other special categories 
of students that are eligible to receive services 
under ESEA Title I included handicapped students 
living in state institutions, American Indian and 
migrant students, and students in state institutions 
for the neglected or delinquent. 

Educationally disadvantaged youth (EDY) funds 
are allocated to school districts pursuant to a 
formula which includes indices of bilinguatism^ 
transiency, and poverty. The California Legislature 
in 1972 provided $82 million for each of three 
years of implementation. Approximately $4.6 mil- 
lion allocated to school districts was used for the 
implementation of EDY only programs, and the 
remaining $77.4 million was used in schools 
receiving other state or federal funds. 

Once their eligibility for EDY funding was 
established, districts selected those school atten- 
dance areas which had students with the greatest 
educational need. Need was determined principally 
by either the number or percent of students 
scoring below the twenty-fifth percentile on stan- 
dardized achievement tests in reading or mathe- 
matics. 

Assumptions of ECE, ESEA Title I, and EDY 
The intent of ECE, ESEA Title 1, and EDY has 
been to provide for increased student achievement, 
particularly in the areas of readings language 
development, and mathematics. While ESEA Title I 
and EDY have addressed selected students who 
have had learning disadvantages, ECE has served all 
kindergarten through third grade students within a 
school. The ECE reform effort has gone beyond 
just changing the instructional program for selected 
students; it creates changes in the institutbn which 
provide a better learning environment for all 
students. Such institutbnal change results from a 



systematic reform of the ways in which the schools 
plan their programs, provide services to students, 
involve parents, utilize community resources, and 
evaluate outcomes for purposes of reptanning. 

It is important to note the assumptions about 
how to effect institutional change which underlie 
the ECE reform movement. Such assumptions are 
the basis of the requirements made of schools and 
districts participating in ECE. 

A m^or assumption of the ECE reform move- 
ment is that the more clearly a school can describe 
what it intends to do. for students and why, the 
greater the probability that such planned activities 
will take place in a timely fashion and the greater 
the likelihood that anticipated results will be 
achieved. Conversely, the greater the degree of 
ambiguity of the school's intent, the lesser the 
degree of timely implementation of planned activi- 
ties and of achieving anticipated results. This type 
of program description is embodied in a program 
plan to be developed at each school by those 
individuals--teaching staff and parents -actively 
involved in the reform effort. 

A second assumption is that program planning as 
well as efforts to implement and evaluate the 
planned program is enhanced by a school based 
advisory committee which broadens the base of 
decision making at the school. Such a school 
advisor/ committee is to be representative of 
parents, community, teaching staff, support per- 
sonnel, and administrators to ensure full considera- 
tion of the various viewpoints of the school 
community, alternative strategies for resolving 
problems, and full use of all available resources. 

A third assumption is that students learn best in 
an individualized program in which there are 
methods of identifying and providing for their 
individual needs and interests. The adult to student 
ratio' must be low to assess adequately and to meet 
such needs and interests. 

A fourth assumption is that the involvement of 
parents is vital: (1) parents are an important source 
of talent for implementing the program; and (2) 
parents' knowledge of their child's classroom 
experience enables them to reinforce at home that 
which the child is learning at school. 

Requifements Relating to Institutional Change 

Districts using a broadly representative district 
advisory committee with responsibility to the local 
governing board were responsible for developing a 
district master plan for ECE; conducting a district- 
wide needs assessment on a school^by^liool basis; 
establishing district program goals and objectives; 



and planning for an orderly phasing-in of the 
district's schools into ECE. 

Schools, with full participation of the represen- 
tative school advisory committees, were required 
to follow the requirements for the schooWevel 
planning process outlined by the state* The plans 
developed through the planning process were to 
Include a needs assessment, school goals and 
measurable performance objectives, appropriate 
solution procedures or activities to close the gap 
between what was and what was desired, a plan for 
both process and product evaluation and feedback, 
a timeline of scheduled events, and a budget which 
showed the coordination of all resoutces in the 
schooL The plan was to consist of program 
components in reading, language development, 
mathematics, multicultural education, services for 
limited and non-English speaking students, staff 
development, parent participation, parent educa- 
tion, health and auxiliary services, and any other 
area which the school believed was appropriate to 
its own situation. 

In the 1974-75 school year, each elementary 
school receiving ECE, ESEA Title I, or EDY funds 
was required to develop a plan for the use of those 
monies. These plans were to cover all funding 
sources in the school. Submission of school level 
plans to the Department of Education was required 



only of ECE schools. Schools funded only by 
ESEA Title I or EDY were not required to submit 
their plans to the Department but were to maintain 
them on file within the school, where a stratified 
random sample was reviewed. 

The Department defined the broad outlines of 
successful program areas in which the needs of 
students were to be assessed. The ECE plans were 
to address, and the programs were to implement, 
the following: individualized, diagnostic instruc- 
tion; staff development and inservice training; 
parent participation; parent education; and health 
needs. Although comprehensive restructuring was 
not inquired in ESEA Title I and EDY, these 
programs were required to individualize instruction 
and to provide staff development, parent partici- 
pation, parent education, and health and auxiliary 
services. 

Requirements Relating to Student Achievement 

ECE, ESEA Title I, and EDY aU stress, as a 
major purpose, increased student achievement in 
basic skills, with primary emphasis on reading, 
language development, and mathematics. In all 
three programs provision was to be made for 
multicultural activities for students and special 
services for students who were non-English or 
limited-English speakers. 
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Procedures, Instrumentation, and Limitations 



In order to evaluate fully the objectives of the 
.urograms included in this summary report, it is 
essential to examine both institutional changes and 
student outcomes. Conclusions based on a review 
of either institutional change or of student out-* 
comes, alone, could be significantly misleading. 
The information to be considered provides both d 
picture of the program in operation and a picture 
f the program's results. By necessity, this report is 
trjsed on a series of measurements or examinations 
jf ^mali facets of the total program. The evaluation 
of the whole program is, therefore, a collection of 
several separate evaluations of parts, which then fit 
together to provide an outline of the whole. 

Institutional Change Data 

Several sources were used to produce data for 
the evaluation of programs receiving ECE, ESEA 
Title I, or EDY funds. While part of the data was 
used to determine participant eligibility, another 
part was for school, district, and state^level plan- 
ning and evaluation. A complete discussion of each 
of the data sources, their use, and their limitations 
is presented in the complete evaluation report. The 
data presented in the findings regarding institu* 
tional change came from the plan ratings, which 
were ratings given to ECE schools' plans submitted 
for review by the Department of Education; the 
monitor and review (MAR) process during which 
Department staff visiting ECE schools rated certain 
items observed at the schools; and the self-^reports, 
which were product evaluation reports and in- 
depth studies in which ECE, ESEA Title I, and 
EDY schools described and evaluated their pro- 
grams. 

Production of the schooblevel plans was, in and 
of itself, evidence of planning and design within 
the local school and was a local self-report of the 
completed needs assessment process. The rating of 
the plans provided a measure of the ways in which 
systematic planning could be- translated to paper. 
This rating was not, however, necessarily an ividica-^ 
tion of the schools' abilities to implement such 
planning. 

School visitation data secured from the MAR 
process provided information on a given school at 
a given time. Such data did not, however, provide 
information about change duriiig the course of a 



year, and the data may have been affected by the 
presence of the observer. Questions of inter-rater 
reliability were also frequently raised in respect to 
such on-site quality judgments. Training sessions 
and field testing of the instrument were carried out 
in 1974-75, but no statistical estimates of the 
reliability of the instrument's application were 
made. 

The self-report information varied from judg- 
ments of the program's quality and effectiveness 
by the people involved to counts of participants 
and monies. Self-^reporting of enumeration data 
was generally quite accurate but had limited 
usefulness beyond suggesting the scope of the 
program. There was no monitoring or auditing of 
the self-reports on accomplishment of objectives 
and activities. Such information, even with the 
assumption of complete good faith, was subject to 
local interpretation. Since there was school-level 
knowledge of the use of these self-reports for the 
evaluation of accomplishment which affected the 
ECE expansion formula, the data may be biased. 
The in-depth study data, however, which were also 
self-^port information, were not used for the 
expansion-formula. One limitation of these data 
may be in their local recommendations. These were 
generated by people closely involved with the 
program who had full knowledge of the situation, 
but who may not have been able to be objective 
about it. 

It can be seen from the foregoing that the 
measurement of institutional change is a very 
difficult process and subject to many limitations. 
When a wide variety of instruments and approaches 
are used, however, and when other measures, such 
as the student outcome data, are combined with 
them, they constitute the most useful tool pres- 
ently available for deriving a collective measure^ 
ment of institutional change and for providing a 
comprehensive picture of the program and its total 
effects. 

Student Achievement Data 

The data presented in the areas of student 
achievement came from objective, norm-referenced 
achievement tests. These tests are relatively insensi* 
tive to specific instructional programs— that is, they 



11 



ERIC 



measure general objectives quite well but measure 
specific objectives poorly or only by inference. The 
instructional activities in any given program fre- 
quently stressed specific objectives and^ hence^ 
may not be adequately measured with norm- 
referenced tests. In such cases, norm-referenced 
tests tend to give underestimates of the actual 
instructional gain made by the students. To com- 
pound this problem, a variety of instruments were 
used to secure data on student achievement. The 
Deportment of Education was constrained^ however^ 
to use norm-referenced tests, since such tests made 
comparisons among groups possible. 

V/hile test scores have conventionally been 
expressed in grade equivalents, many technical 
shortcomings exist in the use of this particular type 
of derived score. This year the Department is 
presenting, in addition to the grade equivalents, a 
standard score which has an arbitrarily defined 
mean of 50 and a standard deviation of 10. The 
presentation of standard scores provides an analysis 
of achievement data that relate the program effects 
during the year to the program outcomes for that 
year. In addition to these characteristics, the 
standard score can be interpreted normatively. For 
example, if students score 48 on the pretest, the 
score represents their relative position to a norma- 
tive group. If those students make a yearns progress 
during the year, their post-test scores would also be 
48. Thus, they have maintained their same relative 
position at post-test time to the normative group* 
To the extent that the post-test score is greater 
than the pretest score, the student or group can be 
Considered as having made greater than a year's 
growth* 



The Department has not attempted to sample 
from the total group of students who were exposed 
to a given instructional component. The test scores 
presented represent all the usable scores submitted 
to the state by participating districts. The number 
of students whose scores are presented is smaller 
than the total number of students in the program. 
This disparity is the result of a variety of factors; 
among them are student transiency, the use of 
inappropriate tests, or other factors which would 
mitigate against the Department's having confi- 
dence in the accuracy of the data submitted. 
Furthermore, only throse students for whom both 
pretext and post-test scores were available are 
included in the analyses. The data used reflect the 
scores of students who received regular and sub- 
stantial services and who were tested with appro- 
priate instruments. Useable test scores were 
analyzed for 54 percent of the participants in 
reading, for 14 percent of the participants in 
language development, and for 51 percent of the 
participants in mathematics. 

A random sample of the data which were 
classified as irregular and unsuitable for processing 
was reexamined, and 50 individual schools were 
contacted in an attempt to determine whether the 
exclusion of these data could have introduced a 
bias in the statewide averages. For some irregu- 
larities, comparable figures could not be discerned. 
In the case of irregular first grade test scores, 
however^ it was found that the average post-test 
scores of schools that did not report pretest scores 
were not significantly different statistically from 
the average post-test scores for schools that did 
report both pre* and post -test data. 
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Program participants inctuded students, parents, 
school personnel, and other community members 
who participated in any ECE reform effort or in an 
ESEA Title 1 or EDY program. Students within a 
participating school who were in various special 
education day classes were not considered program 
participants. A total of 806,752 students in kinder 
garten through grade twelve participated in the 
consolidated programs in 1974-75. (See Figure 1 
for the number of student participants by grade 
level.) Approximately 65 percent of the partici- 
pants were enrolled in kindergarten through grade 
three, with 23 percent in grades four through six 
and 12 percent in grades seven through twelve. 

More students received services in the reading 
component than in any other instructional com- 
ponent. More than 770,000 students were served in 
reading; 580,000, in language development; and 
717,000, in mathematics components. 

The schools maintaining programs employed 
aides, teachers, specialists, and resource personnel. 
Volunteer help constituted a significant contribu- 
tion of time spent in programs by adults. Nearly 
200,000 hours per week of program assistance 
were donated by 67,000 adult volunteers. An 
additional 61,000 students in several grade levels 
volunteered more than 156,000 hours per week in 
assistifig other students. 

Number ol 

student* Number of student pafticlpants strrcd <jn thousands) 
served 

by frada level m jo 50 f.u 7d ku *»o mo ho i2o uio i40 i5o 




F^g. J . Number of student participdnts in ECE» ESEA Title 
1, and EDY, by grade level, 1974^75 



Expenditure Findings 

The allocations, expenditures, and carryover 
funds for each of the three major funding sources 
under the consolidated application process were 
reported by August 15, 1975. Although an audit of 
the reports filed for each funding source was not 
complete at the time this summary report was 
written, it was possible to select for evaluation a 
random sample of reports from ESEA Title I, ECE, 
and EDY schools. Each district reported a sum- 
mary of the total amounts expended for all schools 
within the district receiving the specific funds. (See 
figures 2^ 3, and 4 for the percent of monies 
expended, by categories of expenditures, in the 
districts sampled; however, use caution in the 
interpretation of the data because these figures 
were based on a sample of the reports returned.) 

It should also be kept in mind that several 
funding sources are frequently combined within a 
school, so that the patterns may simply reflect the 
decisions of which source to use to provide specific 
parts of a total local school program. Furthermore, 
the funds in all cases must be supplementary, not 
supplanting. 

institutional Change Findings 

The ECE reform effort provided an '^umbrella'* 
for institutional change within the kindergarten 
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Fig, 2. Budget cat^ortes of ECE fimds )>y peiicent of 
^^x{»eItditttIe» from d landom sample of unaudited 
.ypotts of 22 district summariesi 1974-75 
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Fig> 3, Budfet cat^ories of ESEA Title 1 funds, by percent 
of expenditure, from a random sample of unaudited 
reports of 18 district summaries, 1974^75 




Fig. 4. Budget categories of EDY funds, by peicent of 
expenditure, from a random sample of unaudited 
reports of one cooperative summary and 14district 
summaries, 1974-7S 



through grade three schools. ESEA Title I and 
EDY services for idndergarten through grade three 
students were integrated into the school plan under 
the comprehensive ^^umbreUa/* As a result, the 
overall findings in this section relate directly to 
those schools in which ECE was present; however, 
other funding was present in many cases. 

District level planning occurred, and district 
advisory committees were operational in 1973-74. 
No additional district level plans were requested or 
filed in 1974-75. 



All schools performed school level planning. The 
ECE plans were submitted to the Department of 
Education during the summer of t974 and were 
rated. The plans contained a report of the needs 
assessment process which had been conducted 
prior to the plan development and its results. 

The ways in which needs were identified varied- 
In one school a full day conference was held. All 
members of the staff, many of the parents, and 
some students worked in small groups to discuss 
what they saw the school doing well, what it was 
doing poorly, and what it should be doing. In 
another school, members of the faculty and the 
principal met in homes with different groups of 20 
parents each, holding informal discussions about 
the school. In a third school, a questionnaire was 
sent to all faculty, staff, parents, and students, 
asking them to describe the good and bad things 
that the school was d6ing. 

From the evidence provided by the plan ratings, 
it can be concluded that needs assessment of a 
rather high quality was conducted in approxi- 
mately 75 percent of the schools. The quality of 
the analysis of needs was generally good; in the 
areas of relating the program to the identified 
needs, schools did quite welL 

The development of goals frequently occurred at 
the same time that needs were identified. At local 
schools, groups would develop defined and measut^ 
able performance objectives in different ways. For 
example, one school had a series of evening 
meetings with a group of ten representative parents 
writing goals or objectives statements. Another 
schooFs staff members wrote its objectives; then a 
larg^ group meeting was held for parents during 
which the objectives were either approved or 
modified. In a third situation, a small group of 
parents and staff members went on a weekend 
retreat, wrote the entire plan, and brought it back 
to the other parents and staff members for their 
modification and approval. 

The plan rating data showed that fewer than 20 
percent of the schools had difficulty in making 
clear statements of their goals or desired condi* 
tions, while more than 23 percent were judged as 
excellent. 

Evidence of the implementation of school plans 
and of the effect they had on institutional change 
may be inferred from the on-site reviews and the 
data from the self*reports prepared by the schools. 

CMversity existed not only in the ways in which 
the school plans were developed but atso in the 
implementation of the programs. All programs 
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were to provide an individualized diagnostic, pre- 
scriptive approach to instruction. Within this broad 
outline, the ways in which students were instructed 
varied trom school to school. For example, a 
mathematics and reading test center was estab- 
lished in one school. Each student was tested there, 
and the results were given to the classroom teacher. 
The teacher then established an appropriate two 
week lesson plan for each student, after which the 
students went back to the test center for retesting 
and the cycle was continued. In another school, 
the teacher read every other day with each student, 
made notes about the areas in which the student 
needed help, and then gave a special assignment for 
the next day. In a third school, a classroom aide, 
under the direction of the teacher, assessed each 
student. The teacher reviewed the assessments, 
decided what each student should do next, and had 
a volunteer do that work with the student. 

In terms of individualized instruction, schools 
were rated in reading, language development, and 
mathematics on the quality of their organization, 
diagnosis, prescription and documentation, and the 
continuQus progress nature of their curriculum. In 
genera!, approximately 50 percent of the schools 
were rated "high" in their ability to perform these 
functions in reading and mathematics, with fewer 
than 10 percent below "satisfactory," Although 
more than 75 percent of the schools rated satis- 
factory or better in the area of language develop- 
ment. On several items at least 20 percent rated 
below satisfactory. 

Staff Development and Inservicc Trainii^ 

Restructuring of staff development activities 
occurred not only within schools funded through 
ECE but also in those funded through ESEA Title I 
and EDY. For ECE schools, the monitor and 
review (MAR) ratings were based on the quality of 
staff development to meet the assessed needs of 
the teachers, paid aides, volunteers, and adminis- 
trators and on the involvement of staff in designing 
the program. Diverse staff development activities 
which were systematically related to the program 
occurred at a satisfactory level or above in more 
than 85 percent of the schools. The ability of the 
program to meet the assessed needs of volunteers 
was lower; approximately 70 percent of the schools 
were satisfactory or above in that respect, 

Ftirent Participatioii 

Schools reported that opportunities for parents 
to assess program needs as they perceived them 
were afforded through a variety of methods. 



including participation in formal surveys, school 
advisory committees, site visits, and parent-staff 
conferences. 

Each school planned for the use of its parent 
resources in ways that best met the unique 
circumstances of the school and the community. In 
some schools where there were many working 
parents, participation took the form of assistance 
outside school hours. Other schools were able to 
obtain so many volunteers that they did not need 
as many paid aides as they had originally planned 
to hire. In ECE schools alone, more than 180,000 
parents participated in school programs. 

Th.2 MAR process judged the quality of parent 
participation in ECE schools on the basis of (1) the 
regular meeting and effective representation of the 
parents and the community by the school advisory 
committee; (2) the regular involvement of parents 
in program planning, assistance in classrooms, 
other supportive assistance, and program evalua- 
tion; (3) the existence of an active program to 
arouse parent interest and enlist support; and (4) 
the ability of the program to encourage home- 
school communication in easily understood lan- 
guage. In more than 85 percent of the schools, the 
quality of parent participation was either "satis- 
factory/' "high," or '^exemplary." However, for 
parent involvement in program evaluation activi- 
ties, almost 23 percent of the schools were rated 
less than satisfactory. 

Vmnt Education 

Since parent needs varied considerably among 
ECE schools, there was no one approach to parent 
education. In some areas, for example, the parent 
education program concentrated on basic skills, 
which were identified by the parents as their Tirst 
concern, while in other schools parents requested 
theoretical courses in child development* 

Expansion of the parent education program in 
1974-75 over 1973-74 showed a 55 percent in- 
crease in participants as contrasted to a 30 percent 
increase in participating schools. In the 1,141 ECE 
schools with parent education programs, 118,347 
parents participated, 

Heilth and Auxiliary Services 

The major auxiliary services objectives were 
related to providing pupil personnel and health 
services in ECE, ESEA Title I, and EDY schools. 
The MAR data indicated that in more than 90 
percent of the ECE schools, the quality of the 
pupil personnel and health services was "satisfac- 
tory/' "high," or "exemplary." 
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Pfogram Evahiation tt the Local Level 

All school level plans contained evaluation and 
gemination provisions which were paraUet to the 
activities proposed. Since schools woe required to 
report to the state on the effectiveness of these 
activities at the conclusion of the program^ the 
data submitted to the state were considered evi- 
dence of the evaluation process at the local level. 

Achievement Findings 

This section is designed to give the reader an 
overview of the impact of the various supplemental 
programs on :he achievement of California stu- 
dents who participated in ECE, ESEA Title I, and 
EDY. 

The Reading Component 

A total of 770,000 students were served in the 
reading component of ECE, ESEA Title I, and 
EDY. Usable standard score achievement data were 
available for 313,820 students and are displayed in 
Figure 5. When examining figures expressing stan- 
dard score results, the reader should keep the 
following in mind: A standard score of 50 equals 

Standard scoie 



Numbei of schgob 
by gT«d« l«veL 

Kindergarten -868 



One'-575 
TwO"K53l 
Three -1, 45 2 
Four -905 
Five- I >0 1 8 
Six -947 
Seven--85 
Eight- 82 



0 10 20 30 40 50 60 100 
^ I I > I t J J I 

National 
1 averagic (50) 



^45J^ 



5L2 



49.4 



^44,7 



46.6 



44.2 



43.5 



ML 



All. 



42.5 



42.2 
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Fig. 5. Avetiv^ pretest and post-test staitdird scores in 
leadiiig tchievementf by grade levels for schools that 
received ECE, ESEA Title I, md EDY Auids in 
1974^75 



the national average. Looking at Figure 5, the 
reader will note that the pretest scores for kinder- 
gartens averaged 44.4. The post-test score was 
51.2. This represented 6.8 points more gain than 
would have been expected in one year of instruc- 
tion. It also indicated that kindergarten students 
exceeded the national average in reading achieve^ 
ment on the post-test. It can also be seen that 
students in grade one approximated the national 
average in reading achievement on the post-test. 
Students in grades two and three showed gains, but 
the post-test was somewhat lower than that for 
kindergarten and grade one— and, in fact, con- 
tinued to decrease through grade eight. While the 
post-test scores showed that students in the upper 
elementary grades remained below the national 
average, there was an absolute magnitude of gain 
evident across aU grades for all students measured. 
This g^n was greatest in the lower grades and least 
in the upper grades. 

Figure 6 shows essentially the same information 
as that presented in Figure 5, except it is expressed 
as gkade equivalents, and it includes results for 
secondary schools. Again, growth is evident for all 
students tested in aU jgrades. The greatest growth 
was in the primary grades and at the secondary 
level. The least amount of growth, although it is 
still substantial, was in the upper elementary 
grades. The scores reported as grade equivalents for 
the high school level were based on a small number 
of students. Therefore, while the growth rate was 
very substantial in grades eleven and twelve, it 
must be interpreted with caution. 

The Mathematics Component 

More than 717,000 students were served in the 
mathematics component in ECE, ESEA Title I, and 
EDY. Usable standard score data were available for 
276,371 students. Figure 7 shows achievement 
gains in mathematics. While students in grade one 
were at the national average on the post-test, 
students in kindergarten and grade two were 
slightly below the national average. Progressing 
through the grades, the post-test levels continue to 
decline; yet, the post-test scores were higher than 
the pretest scores for the same students at all grade 
levels. 

When data for the mathematics component are 
presented as grade equivalent scores (Figure 8)^ 
similar growth patterns are evident: greatest 
growth in early elementary grades and least growth 
in grades seven through nine. The growth rates in 
grades ten and twelve should be interpreted with 
caution due to the small number involved. 
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Fit* 6, Avenge pvetest ind post*test grade equivtlent 
scores In reading achievement, gnde level, for 
sclioob ibwt received £CE, ESEA Tide 1, and EDY 
funds for 1974^75 

Findingg from Early Childhood Education 

Since the BCE reform effort was implemented in 
a relatively comprehensive manner^ it was appropri'' 
ate to review the outcomes specifically of ECE 
schools in the areas of reading and mathematics, 

Reading achievement data for ECE students in 
schools having various combinations of funding 
sources are presented as standard scores in Figure 
9, A review of the figure shows that students in all 
combinations of ECE program^ had post-test scores 
in kindergarten and grade one tl^t exceed the 
national average (50X while students in grades two 
and three were closer to the national average on 
the post-test than they were on the pretest. 
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43.7 



i41.3'; 



44:1 



MM 



42.9 



42.2 



Kirtiicrtiartui 812 
One 556 
Two L340 
Tlircc 
Four *J08 
Five L015 
Six- 943 
Seven -87 
Eight -80 
Ninu-46 
Ten- 16 



F^ 7, Average pretest and post-test standard scoies in 
mathentaCics achievement, grade tevel, for schools 
that received ECE* USEK Tide 1, and EDY funds in 
1974^75 



A further review of Figure 9 shows tliat the 
program combinations of ECE/Miller^'Unruh and 
ECE/Title I yielded the highest post-test scores for 
students in kindergarten through grade three. 

The Department used reading achievement 
scores from the California assessment program 
(CAP) as an additional check on state evaluation of 
pfog^ms, AU ECE schools were compared with 
non-ECE schools in terms of four indices: (1) 
average socioeconomic status; (2) number of grade 
three students; (3) total percentage minority en- 
rollment; and (4) 1974-75 third grade predicted 
score on the CAP reading achievement test. The 
frequency distribution of ECE and non-ECE 
schools across the four indices is shown in the 
appendix to the complete evaluation report A 
great deal of variation exists among both ECE and 
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achievement test taken as third gradeis in 1974^75. 
As seen in Table 2» students in both one and two 
year ECE schools showed statistically signincant 
higher gain scores than students in non-ECE 
schools. 

Since two*year ECE ixhools showed slightly 
greater g^ins than one year ECE schools (15,7 vs-, 
15.5), this is viewed as a positive result for the ECE" 
program for at least two reasons, Fiist^ it puts to 
rest the contention that ECE would have only a 
temporary effect that would vanish in the second 
year of operation. Instead, there appears to be a 
cumulative effect over years. Secondly, the two- 
year jtiCE schools have a somewhat lower hidex of 
socioeconomic status than the one year schools, 
which indicates that their gains would generally 
have been predicted to be smaller than those of the 
one-year ECE schools. 

Table 2 

Reading Achievement Cain Scoies for ECE Schools 
and Matched Group of Non-ECE Schools 



Type of school 


Third gride 
■cpreSf 
1974-73 


Second 
grade 

loareSt 
1973-74 


Gahi scores* 


Two years in ECE 
(N = 427) 


79.2 


63,5 


15.7t 


One year in ECE 
(N = 658) 


sto 


65,5 


15.5t 


Matched non*ECE 
schools 
(N =3.326) 


80,6 


65.6 


15.0^ 



*These scores are presented is number concct in 1974-75 less 
numbei conect in 1*>73*74, 



non-ECE schools. Yet, it can be seen that com* 
pared to non-ECE schools, the ECE schools on the 
average had indices of lower socioeconomic status, 
larger school size, and a higher percentage minority 
enrollment. 

Longitudinal profiles of student performance in 
reading achievement were computed for ECE 
schools in the program for one year, ECE schools 
in the program for two years, and a matched group 
of non-ECE schools, Readhig achievement gain 
scores from CAP, as presented in Table 2, follow 
the students in the same schools for two years; 
1973-74 and 1974-75, The reading achievement 
scores made by second graders in 1973'*74 were 
subtracted from the scores on an identical reading 



A comparison of grade equivalent gains in 
reading and mathematics was made between first 
year ECE schools in 1973-74 and first year ECE 
schools in 1974-75, Table 3 shows the comparisons 
of gains for the respective first year schools. The 
data presented indicate that during their first year 
of operation, schools entering ECE in 1974-75 
showed significantly greater gains than did schools 
in their first year of operation in 1973-74, Since it 
was not possible to match these schools, and the 
schools reporting in grade equivalent scores in 
1974-75 represented less than half the total num- 
ber of schools reporting, caution should be used in 
the interpretation. 
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The data in Table 3 appear to contradict the 
argument that the first group of schools in ECE 
were such a select, highly motivated group that the 
"Hawthorne effect'* was responsible for their gains 
in the flrst year, and^ thus, subsequent years 
schools' scores would therefore show lower 
achievement gains. 

Standard score achievement data in mathematics 
indicated that ECE students in kindergarten and 
grade one exceeded the national average (50) on 
the post-test, while students in grades two and 
three were less than one point below the national 
average (see Figure 10). In all cases, post-test scores 
were hi^er than pretest scores. 

Students in schools having ECE monies (only) or 
a combination of ECE/Title I monies exceeded the 
average for all other combinations of ECE funded 
schools at all grade levels (kindergarten through 
grade three) in mathematics. 



Tabie 3 



Average Increase ib Grade Equivalent Scores in Reading 
and Mathematics Between Pretest and Post-test 
for ECE Sdiools in Their First Year ^ Operation 



SkiUand 
grade 


1973-74 gsdm 
for schoob 
entering ECE 
in 1973-74* 
N = 814 


1974*75 gains 
for schools 
entering ECE 
in 1974-75* 
N = 49l 


Level of 
significance 


Reading 








Grade one 


•75 


•98 


•001 


Gnde two 


.83 


•95 


•001 


Gnde three 


.83 


L04 


•001 


Nbthemattcs 








Gnde one 


•84 


L07 


•001 


Grade two 


•91 


•94 


N/S 


Grade three 


•99 


LOS 


•016 



^These grade eqnivaleitt gains reflect absolute magiiilitde of change 
and are not extrapolated to lOmonths of instruction^ 
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*NOTE: Miller-Unnih funding was present in these schools^ 
and such funding was used only for reading. 

Fig. 9. (continued) Weighted average pnetest and post-test standaid scores in leading achievement^ by 
grade level, for schools ptrtfcipathig fat early childhood educatlcm, 1974^75 
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level, for Khoots participating In tarty cUMdood edvcatlmi, 1974'75 
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Fig. 10* (continued) Weighted avenge pretest and post-test stmdinl scores in mathematics achieve- 
ment, by girade level, for schools participating in early childhood education, 1974^75 
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